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Abstract 

In  this  paper  refined  large  deviation  asymptotics  are  derived  for  the 
classical  occupancy  problem.  The  asymptotics  are  established  for  a 
sequential  filling  experiment  and  an  occupancy  experiment.  In  the  first 
case  the  random  variable  of  interest  is  the  number  of  balls  required  to 
fill  a  given  fraction  of  the  urns,  while  in  the  second  a  fixed  number  of 
balls  are  thrown  and  random  variable  is  the  fraction  of  nonempty  urns. 


1  Introduction 

The  classical  occupancy  problem  [8]  is  concerned  with  the  number  of  oc¬ 
cupied  urns  after  r  balls  have  been  thrown  into  n  urns,  and  with  the  balls 
thrown  according  to  Maxwell-Boltzmann  (MB)  statistics  (i.e.,  each  ball  en¬ 
ters  any  urn  with  equal  probability,  and  different  throws  are  independent  of 
one  another).  It  is  a  fundamental  model  that  appears  in  many  contexts. 

The  occupancy  problem  can  be  regarded  from  two  related  points  of  view. 
One  perspective  focuses  on  the  filling  process  and  the  other  on  the  occupancy 
process.  In  the  filling  process,  balls  are  thrown  in  an  endless  sequence  and  we 
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record  the  number  of  balls  that  must  be  thrown  before  a  previously  empty 
urn  becomes  nonempty.  This  produces  a  sequence  of  integer  valued  random 
variables  whose  sum  is  the  random  number  of  balls  needed  occupy  a  given 
number  of  urns.  In  the  occupancy  process,  a  hxed  number  of  balls  are  thrown 
and  we  record  the  fraction  of  occupied  urns  after  each  ball  is  thrown.  This 
produces  a  sequence  of  random  variables  that  is  monotonically  increasing, 
with  the  last  variable  representing  the  random  fraction  of  occupied  urns 
after  all  balls  are  thrown. 

Clearly  these  two  processes  are  closely  related,  in  that  one  is  essentially 
the  inverse  of  the  other.  Indeed,  it  is  tempting  to  think  they  give  exactly  the 
same  information  and  that  statements  on  the  asymptotic  behavior  of  one 
immediately  translate  into  statements  on  the  asymptotic  behavior  of  the 
other.  Although  this  would  be  true  if  the  processes  were  strictly  monotonic, 
in  fact  they  are  not.  The  differences  between  the  information  contained  in 
the  processes  is  made  precise  in  Section  2.2.  We  will  see  that  once  these 
differences  are  accounted  for,  it  is  still  possible  to  analyze  the  behavior  each 
model  once  one  understands  the  other. 

For  the  occupancy  process,  process  level  large  deviation  principle  (LDP) 
results  for  the  fraction  of  occupied  urns  are  given  in  [19],  together  with  the 
solution  to  the  associated  calculus  of  variations  problem  when  the  termi¬ 
nal  value  of  the  trajectory  is  hxed  (this  identihes  the  rate  function  for  the 
outcome  of  the  occupancy  experiment).  The  papers  [7,  4]  generalize  the 
process  level  LDP  determined  in  [19],  and  moreover  [7]  obtains  strong  min¬ 
imizing  extremals  to  the  associated  calculus  of  variations  problem  in  this 
more  general  setting. 

On  the  other  hand,  the  paper  [16]  directly  studies  the  hlling  process. 
Using  the  Gartner-Ellis  Theorem  [6]  and  a  representation  for  the  number  of 
balls  needed  to  hll  a  given  fraction  of  urns  (the  hlling  experiment)  as  a  sum 
of  random  variables,  [16]  proves  an  LDP  for  the  ratio  of  this  number  to  the 
number  of  urns. 

The  analysis  of  the  occupancy  process,  as  in  [7],  provides  greater  qual¬ 
itative  insight  into  the  behavior  of  occupancy  problems.  However,  because 
the  hlling  process  has  this  very  convenient  interpretation  as  a  sum  of  inde¬ 
pendent  random  variables,  it  is  a  more  natural  object  to  study  when  consid¬ 
ering  asymptotics  that  are  more  rehned  than  just  large  deviation  properties. 
Hence  in  this  paper  we  will  hrst  focus  on  getting  the  rehned  large  deviation 
asymptotics  for  the  hlling  process. 

Throughout  this  paper  the  same  asymptotic  scaling  as  in  [7]  will  be 
used.  Let  [aj  denote  the  integer  part  of  a  G  [0,oo).  Fix  6  G  (0,oo),  let 
r  =  [6n\  and  consider  the  limit  as  n  ^  oo.  Suppose  that  Fg(0)  denotes 
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the  fraction  of  urns  that  are  empty  after  all  r  balls  have  been  thrown.  It 
is  easy  to  derive  a  law  of  large  numbers  limit,  and  indeed  the  fraction  of 
unoccupied  urns  converges  in  probability  to  by  the  well  known  Poisson 
approximation  [14],  In  this  context,  1— ro(0)  >  ^  >  1  — corresponds  to  a 
rare  event  (exceptionally  many  occupied  urns),  and  likewise  1  —  Pq  (0)  <  ^  < 
1  —  constitutes  a  rare  event  (exceptionally  many  empty  urns).  Our  main 
results,  which  are  stated  in  Section  2,  are  explicit  higher  order  asymptotic 
approximations  for  the  probabilities  of  such  rare  events. 

For  classical  occupancy  models,  combinatorial  formulas  for  certain  prob¬ 
abilities  can  be  obtained  using  the  inclusion-exclusion  principle.  However, 
since  in  the  setting  of  rare  event  problems  one  must  add  and  subtract  quan¬ 
tities  that  are  large  compared  to  the  quantity  one  hopes  to  compute,  these 
“exact”  formulas  are  not  always  useful.  Indeed,  for  some  of  the  calculations 
given  at  the  end  of  the  paper  the  exact  formulas  did  not  give  a  meaningful 
answer.  Also,  because  they  are  not  analytic  expressions  the  formulas  do 
not  provide  much  in  the  way  of  qualitative  insight.  Asymptotic  approxi¬ 
mations,  and  in  particular  large  deviation  approximations,  often  provide  a 
more  useful  alternative. 

The  classical  occupancy  problem  arises  in  several  applications.  For  ex¬ 
ample,  in  complexity  theory  it  appears  in  connection  with  the  random  3- 
SAT  problem  [13].  Here  there  are  n  variables,  and  formulas  consisting  of 
cn  boolean  clauses  with  3  distinct  variables  are  chosen  at  random  according 
to  a  uniform  distribution.  If  c  is  too  large,  no  value  of  the  variables  will 
satisfy  the  formula,  i.e.,  make  its  value  true,  with  high  probability.  In  [13]  it 
is  shown  that  if  c  >  4.76  this  is  the  case  by  using  a  large  deviations  analysis 
of  the  classical  occupancy  model  originated  by  Weiss  [19].  In  this  model  the 
variables  are  taken  as  the  urns  and  a  certain  type  of  clause  (one  with  exactly 
one  unnegated  variable)  are  taken  as  the  balls. 

In  connection  with  statistical  hypothesis  testing,  large  deviation  approx¬ 
imations  (and  in  particular  refined  approximations  such  as  those  we  describe 
later  on)  can  be  used  to  construct  confidence  intervals  for  tail  probabilities 
for  which  the  central  limit  theorem  might  give  poor  estimates.  For  example, 
in  [5]  the  problem  is  to  determine  how  many  sensors  are  active  in  a  network. 
One  keeps  a  count  of  those  which  have  responded  to  query  signals.  Suppose 
each  response  confirms  the  activity  of  one  randomly  chosen  sensor  and  it  is 
desired  to  estimate  the  number  of  the  active  sensors  with  high  probability. 
Estimators,  confidence  intervals,  and  estimates  for  error  probabilities  can 
all  be  constructed  for  networks  with  a  hundred  or  more  (active)  nodes  using 
the  refined  large  deviation  asymptotics  described  here. 

As  a  final  example  large  deviation  approximations  for  the  classical  oc- 
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cupancy  model  can  also  be  used  to  dimension  optical  switches.  Here  the 
problem  is  to  determine  the  number  of  shared  any-color  to  any-color  wave¬ 
length  converters  that  would  be  needed  to  provide  satisfactory  transmission 
in  a  bufferless  optical  packet  switch.  Other  applications  include  data  bases 

[I] ,  and  more  recently  coding  theory  [15]. 

There  is  a  small  collection  of  papers  that  study  the  sort  of  “higher  or¬ 
der”  large  deviation  approximations  considered  in  the  present  paper.  The 
first  work  in  this  area  is  Bahadur  and  Rao  [3],  which  considers  the  sample 
mean  for  independent  and  identically  distributed  random  variables.  litis 

[II]  considers  higher  order  large  deviation  asymptotics  for  Markov-additive 
random  variables  in  where  the  chain  is  regenerative,  aperiodic  and  time 
homogenous.  In  both  papers,  a  refined  central  limit  approximation  is  ap¬ 
plied  to  the  twisted  distribution,  which  in  the  case  of  [11]  uses  techniques 
similar  to  those  in  [12].  Also  in  both  papers  the  underlying  processes  are 
time  homogeneous,  which  differs  from  the  state  dependency  present  in  the 
evolution  of  the  occupancy  process. 

A  paper  which  proves  higher  order  approximations  for  processes  with 
state  dependency  is  Azencott  [2].  This  paper  considers  solutions  of  “small 
noise”  stochastic  differential  equations  and  gives  refined  approximations  for 
probabilities  of  sets  of  trajectories.  These  sets  of  trajectories  must  satisfy  a 
certain  smoothness  condition  on  the  boundary  of  the  set.  The  corresponding 
central  limit  approximations  are  somewhat  more  straightforward  in  that  the 
original  process  is  defined  in  terms  of  a  Gaussian  driving  noise.  Although 
the  rate  coefficients  are  not  given  in  explicit  form,  the  structure  of  the  ap¬ 
proximation  is  similar  to  one  that  we  obtain  for  a  certain  parameter  regime 
in  the  occupancy  problem. 

Lastly  we  note  that  Fleming  and  James  [10]  also  investigate  higher  order 
asymptotics  for  the  probability  that  a  “small  noise”  diffusion  process  exits 
a  fixed  domain  before  a  given  fixed  time.  However,  the  results  of  [10]  apply 
only  to  initial  conditions  for  which  the  large  deviation  minimal  cost  trajec¬ 
tory  exits  before  the  given  terminal  time.  Although  some  of  the  problems 
we  consider  could  be  formulated  as  such  “exit  time”  problems,  it  turns  out 
that  the  minimal  cost  trajectory  will  always  exit  exactly  at  the  terminal 
time,  and  so  the  methods  used  in  [10]  do  not  seem  to  apply. 

In  terms  of  technique,  our  approach  is  closest  to  that  of  [3] .  However,  the 
final  form  of  the  result  is  qualitatively  quite  different  from  that  of  [3],  and 
this  is  due  to  both  the  state  dependencies  and  to  certain  natural  boundaries 
on  the  state  space  of  the  occupancy  process  (one  cannot  have  more  filled 
urns  than  the  total  number  of  urns  available). 

In  what  follows  we  omit  the  case  where  the  urns  are  not  all  empty  ini- 
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tially,  which  requires  only  minor  modifications  of  the  methods  that  we  use. 
We  expect  that  these  methods  would  also  apply  with  only  small  adjustments 
to  the  case  when  balls  “miss  the  urns”  with  a  fixed  probability  p  >  0  inde¬ 
pendently  at  each  trial  [18],  but  have  not  verified  all  details  at  the  present 
time.  A  more  substantial  generalization  is  to  consider  statistics  other  than 
Maxwell-Boltzmann,  such  as  Bose-Einstein  or  Fermi-Dirac.  Another  inter¬ 
esting  generalization  is  refined  large  deviation  approximations  for  the  ran¬ 
dom  vector  whose  fth  component  is  the  fraction  of  urns  containing  exactly 

1  balls,  i  <  I  for  a  fixed  constant  I. 

The  rest  of  the  paper  is  organized  as  follows.  In  Section  2  we  define  the 
probability  model  and  review  known  large  deviation  results  for  the  classical 
occupancy  problem.  These  include  a  description  of  an  importance  sampling 
scheme  which  allows  accurate  empirical  estimation  of  rare  event  probabilities 
in  classical  occupancy,  which  will  be  used  later  when  we  present  data  on 
these  various  approximations.  We  then  state  the  main  results.  The  proof 
is  given  in  Section  3.  It  relies  on  a  refined  central  limit  approximation  for 
sums  of  independent  but  non-identically  distributed  random  variables  lying 
on  a  common  lattice,  and  generalizes  several  results  in  [9,  Chapter  XVI]. 
Finally,  in  Section  4  we  present  some  numerical  results  for  the  asymptotics 
as  well  as  approximate  values  obtained  using  an  “exact”  approach  and  the 
importance  sampling  scheme. 

2  Review  and  Main  Results 

2.1  Large  Deviations  for  the  Occupancy  Process 

We  restrict  attention  to  occupancy  models  with  Maxwell-Boltzmann  sta¬ 
tistics.  In  [7,  4]  a  large  deviations  principle  was  proved  occupancy  models 
with  the  number  of  urns  n  as  the  scale  parameter  and  the  number  of  balls 
r  =  \n9\  in  fixed  proportion  as  n  — >  oo.  [7]  includes  a  fairly  complete  so¬ 
lution  to  the  associated  calculus  of  variations  problem  that  must  be  solved 
to  obtain  the  large  deviations  exponent.  In  what  follows  we  restrict  to  the 
case  of  starting  with  all  urns  initially  empty,  the  case  where  some  urns  are 
already  occupied  being  a  straightforward  extension.  Suppose  that  a  total 
of  i  balls  (or  t  =  i/n  balls  per  urn)  have  been  thrown  at  some  stage  in  the 
experiment.  Then  the  fraction  of  occupied  urns  rQ^(t)  performs  a  random 
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walk,  with  rQ^(O)  =  0  and 


=  r^+(t),  w.p.T'S^t) 

=  ro^(t)  H — ,  otherwise. 


By  the  well  known  Poisson  approximation  [14] ,  after  nO  balls  have  been 
thrown  (or  9  balls  per  urn),  the  fraction  of  empty  urns  is  approximately  e“®. 
Thus  there  are  two  different  rare  events  of  interest: 

Vi  =  n+(0)>e:e<0,e>l-e-n, 

V2  =  {r^+(0)<e:e<l-e-^}.  ^  ^ 

Vi  corresponds  to  the  rare  event  that  exceptionally  many  urns  are  occupied 
and  V2  corresponds  to  the  rare  event  that  exceptionally  few  are  occupied. 
In  what  follows  we  assume  that  and  n6  are  both  nonnegative  integers. 


Remark  2.1.  Note  that  in  the  ease  of  event  V2  9  >  log(l  —  ^  In  the 
ease  of  event  Vi  we  impose  the  eondition  f,  <  9,  but  there  is  no  real  loss  of 
generality.  If  >  9  then  we  need  more  nonempty  urns  than  there  are  balls, 
whieh  is  impossible  (i.e.,  P{rg^(0)  >  ^}  =  Oj.  If  =  9  then  {ro+(0)  > 
eorresponds  to  the  event  that  every  ball  falls  into  an  empty  urn,  an  event 
whose  probability  is  •  Note  also  that  when  =  1,  Vi  is  a  rare  event 

for  any  I  <  9  <  00,  and  that  there  is  no  V2  type  rare  event.  In  this  ease,  the 
event  Vi  means  that  every  urn  is  oeeupied  after  n9  balls  have  been  thrown. 

Let  Q(Z_|_)  denote  the  set  of  probability  measures  on  the  non- negative 
integers,  and  define  the  relative  entropy  of  /r  G  with  respect  to  z/  G 

Q(Z+)  by 

D{p\\y)=  ^  \og{pi/vi)pi 

*6  + 

(with  the  understanding  that  OlogO  =  0).  The  large  deviations  exponent 
for  both  the  events  described  above  is  given  by  (cf.  [7,  16]) 


.J{9)  =  inf  71  (T  11  V{9)), 

where  the  infimum  is  over  all  T  G  Q(Z+)  subject  to  Tg  =  1  —  ^  and  iTj  = 
9,  and  V{9)  denotes  the  Poisson  distribution  with  parameter  9.  The  infimum 
is  readily  found  to  be 

J{9)  =  {9-  ^)logp-h  (1  -  C)log(l  -  0  -  — ^log(l  -  pi),  (2.2) 

P 
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where  p  is  the  unique  positive  root  to 

Op  =  -  \og{l-  pi).  (2.3) 

To  see  that  such  a  root  exists  and  is  unique,  first  note  that  0  is  always  a  root. 
The  right  hand  side  of  (2.3)  is  a  strictly  convex  function,  with  derivative  with 
respect  to  p  equal  to  ^  at  p  =  0,  and  which  increases  to  oo  as  p  t  Hence 
a  unique  positive  solution  to  (2.3)  always  exists  provided  9  >  i-  Moreover, 
when  it  does  exist,  p.^  <  1. 

As  is  well  known  in  large  deviation  theory,  the  asymptotically  most  likely 
trajectory  for  the  random  walk,  conditioned  on  the  outcome  of  either  exper¬ 
iment  (for  example  Vi),  can  be  identified  as  the  “cheapest  cost  trajectory” 
for  an  associated  calculus  of  variations  problem.  In  the  present  context  this 
is  just  the  absolutely  continuous  trajectory  ipQ  that  minimizes 

D  II  {i/joit),!  -  'i/’o(t)))  dt 

subject  to  the  constraints  "00(0)  =  1  and  V’o(^)  <  1  —  and  with  D  now 
relative  entropy  for  probability  measures  on  {0, 1}.  As  the  results  in  [7] 
show,  the  minimizer  is  just 

p  \  pj 

The  parameter  p  is  useful  in  change-of-measure  importance  sampling  [17]  to 
obtain  empirical  estimates  for  rare  event  probabilities  in  classical  occupancy. 
An  efficient  scheme  can  be  obtained  by  multiplying  the  probability  that  a 
ball  enters  an  occupied  urn  by  p,  i.e.,  to  consider  a  new  measure  under  which 

P {rs+  (‘  +  ^)  =  ro+{<)}  =  pr„\(() , 

p  can  be  interpreted  as  a  “twist  parameter.”  When  p  <  1  (which  is  true  in 
the  case  ^  >  1  —  e~^)  we  make  it  more  likely  that  balls  fall  into  the  empty 
urns.  When  p  >  1  (true  in  the  case  ^  <  1  —  e~^)  we  make  it  more  likely  that 
balls  fall  into  occupied  urns. 

We  now  go  over  some  preliminaries  for  the  filling  experiment  before  sta¬ 
ting  the  main  results. 

2.2  Filling  a  Given  Fraction  i  of  Urns 

The  random  number  of  balls  needed  to  fill  a  given  fraction  i  of  the  urns  is  the 
sum  of  ni  non-identical  independent  random  variables.  The  zth  summand 
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represents  the  number  of  balls  that  are  thrown  to  occupy  an  empty  urn 
when  i  —  \  urns  are  already  occupied.  By  a  failure  probability  we  mean 
the  probability  that  a  thrown  ball  lands  in  an  occupied  urn,  and  so  the 
failure  probability  at  time  i  is  qf  =  {i  —  l)/n.  Suppose  Xf  is  the  number 
of  balls  thrown  to  occupy  another  empty  urn  immediately  after  i  —  1  urns 
have  become  occupied.  Dehning  Gk{q)  =  q^~^P,  k  =  1,2, . . .  ,p  =  I  —  q,  with 
p2  =  1  —  q^,  we  have 

F{Xf  =  k}  =  Gk{q?), 

En-i  vn 
^  ^  _  i=l 

n 

In  terms  of  occupancy  process,  the  number  of  balls  per  urn  needed 

to  occupy  a  given  fraction  ^  G  (0, 1]  of  the  urns  satishes 

F-(e)=min{t:r^+(t)>e}. 

By  the  Poisson  approximation,  k,  —  log(l  —  ff)  balls  per  urn  are  re¬ 

quired  to  occupy  urns.  Two  different  kinds  of  rare  events  connected  with 
the  hlling  process  are 

Wi={F"(e)<0:e<0<-log(l-e)} 

W2  =  {Y^{i)>e-.e>-\og{i-i)] 

=  >  0  :  e  <  1  -  e-®}. 

Note  that  when  ^  =  1  Wi  is  a  rare  event  for  any  1  <  0  <  oo  (and  there  is 
no  type  W2  rare  event).  Wi  in  this  case  should  be  interpreted  as  using  no 
more  than  9  balls  per  urn  to  occupy  all  the  urns. 

Clearly  Vi,  V2  as  dehned  in  (2.1)  are  closely  related  to  Wi,  W2.  In  fact 

Vi  =  Wi.  (2.4) 

However  V2  7^  W2.  Suppose  that  W2  has  occurred.  Then  at  least  9  balls  per 
urn  were  required  to  hll  urns,  which  implies  after  throwing  9  balls  per 
urn,  there  must  be  at  most  nf,  urns  occupied,  i.e., 

r^+(0)<e, 


so  V2  has  occurred. 

However,  V2  can  occur  even  when  W2  does  not  occur.  Suppose  that 
urns  are  occupied  strictly  before  n9  balls  are  thrown,  but  that  all  additional 
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balls  (up  till  and  including  when  ball  nO  is  thrown)  fall  into  already  occupied 
urns.  Then  V2  occurs,  though  yV’2  does  not.  We  can  formulate  this  as 

V2  =  W2  U  {y”(e)  <  e  and  >ne-  nY^{i)]  .  (2.5) 

Let  Fn{a)  =  ^  log  IE  be  the  scaled  logmoment  generating  function. 

Then  one  can  readily  compute 

Fn{a)  =  a^+ 

^  i=i  ^  j=i 

Fix  a  <  —  log^.  Interpreting  the  last  display  as  a  Riemann  sum,  letting 
n  — >  00,  and  then  evaluating  the  resulting  integral  shows  that  Fn{a)  — > 
F(a),  where 

F{a)  =  -  (1  -  ^)  log(l  -  0  +  ^  ^  ■  (2-^) 

Note  that  the  convergence  of  F^  to  F  is  true  for  all  values  of  including 
^  =  1.  When  ^  =  1  we  restrict  to  a  <  0,  and  F{a)  =  a  +  log(l  —  e"). 

Define  the  function  J„(0)  to  be  the  Legendre  transform  of  the  log  moment 
generating  function: 


Jn(6')  =  supja^  -  Fn{a)}.  (2.8) 

a 

Direct  calculations  show  that  (i)  is  strictly  convex  on  its  domain  of 
finiteness,  (ii)  F^{—oo)  =  and  (hi)  that  F^ia)  j  00  as  a  j  —  log[(n^— l)/n]. 
It  then  follows  that  the  supremum  in  the  last  display  is  finite  and  attained 
at  some  unique  a*  <  —  log[(n^  —  l)/n]  whenever  9  >  with  9  =  F))(a*). 
Also  observe  that  F^{0)  «  —  log(l  —  ^).  Hence  for  sufficiently  large  n,  if 
^  >  1  —  then  a*  <  0  and  if  ^  <  1  —  then  a*  >  0.  Define 

p„=expa*.  (2.9) 

Then  9  =  F^{an)  becomes 


Define 


J{9)  =  sup{a0  —  F{a)} 

a 

=  eiogp- F(iog(p)), 
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where  a*  =  log  p  uniquely  achieves  the  supremum.  Then  simple  calculation 
shows  that  p  indeed  satisfies  (2.3).  Inserting  the  expression  for  F  we  recover 
the  equation  (2.2),  and  thus  J  =  J.  Again  notice  that  this  holds  for  both 
cases  ^  =  1 ,  ^  <  1 . 

Later  on,  we  will  prove  that  Pn  ^  p  and  a*  — >  a*.  Define  the  “twisted 
variance” 


= 

^ n  ^  n  V^n) 


n 


ni 

E 


i=i  (l  - 


(2.11) 


Again  using  the  standard  Riemann  integral  approximation  and  the  bound 
a*  <  —  log^,  we  have  where 


=  F"(a*) 


(1  -  te"*)^ 


dt. 


Integrating  gives 


^2  = 


1-pC 


(2.12) 


2.3  Main  Results 

The  main  results  of  this  paper  can  now  be  stated.  We  will  first  give  the 
refined  asymptotics  for  the  filling  process  and  then,  by  incorporating  the 
difference  (2.5),  analogous  results  for  the  occupancy  process.  We  first  make 
a  definition. 

Definition  2.1.  Consider  a  sequence  of  numbers  Pn  G  [0, 1]  and  J  G  [0,  oo] 
such  that  ^ 

-  log  Pn  J- 
n 

Then  K  G  (0,  oo)  is  a  v-prefactor  for  {pn}  if 

rCpne'"-^  K. 

Theorem  2.1.  //^  <  1,^  <  0  and  ^>1  —  then  Pn  =1P{E”(C)  <  = 

P{VVi}  has  a  ^-pref actor  with 

lim  yE  P  {^“{O  <  0)  =  -5^  ( -G_)  (2.13) 

n^oo  V27ra^  \l- -  Pj  \  1-? 
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If  C  <  I  and  C  <  I  -  e-^  then  =  P{y"(e)  >  9}  =  P{W2}  has  a  i- 
prefactor  with 


lim  ^AIP{F"({)>e}e"''«»=-^(^'),/^.  (2,14) 

n— >oo  V27r<7^  \P~^/  V 

//  ^  =  1,  ^  <  0,  Pi'j  =  P  {y"(l)  <  0}  =  P  {>V2}  has  a  0-prefactor  with 

lim  P{y”(l)  <  9}  ^  (2.15) 

n^oo 

In  all  cases  p  is  the  unique  positive  root  of 

9  =  --log(l  -  pC) 

P 

and  (7^  is  determined  as  in  (2.12). 

Theorem  2.2.  Iff,  <1,  f  <9  and  f  >  1  — e“®  then  ql^  =P{rQ^(0)  >  ^}  = 
P{Vi}  has  a  ^-pref actor  with 


lim  ^  P  {rS,(.)  >  f}  =  ^  (^)  (2.16) 

If  f  <  I  and  f  <  I  —  e~^  then  qn  =  P{rQ^(0)  <  ^}  =  P{V2}  has  a 
prefactor  with 


lim  y/fi  P  {r((+(0)  <  f}  = 


V2 


Tra^ 


P 


+ 


p  -  1  1  - 


'1-pe 


If  f  =  1,  qn  =F  {rQ^(0)  =  l}  =  P{Vi}  has  a  0-prefactor  with 

1 


lim  P{r((+(0)  =  l}e 


—  1  \  f,nJi9)  — 


1-e  ■ 

(2.17) 


(2.18) 


In  all  cases  p,  <7^  are  obtained  as  in  Theorem  2.1. 

Owing  to  (2.4)  the  ^-prefactors  for  and  qln  are  the  same,  as  are  the 
0-prefactors  for  Pn  and  q\.  However,  as  suggested  by  (2.5),  the  ^-prefactor 
for  qn  differs  from  that  of  pn  • 
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3  Proof  of  the  Main  Results 


3.1  An  Extension  of  the  Central  Limit  Theorem 

In  what  follows  we  will  need  the  following  theorem,  which  generalizes  [9, 
Page  540,  Theorem  2]  to  independent,  non-identical  lattice  random  vari¬ 
ables.  Let  {Wp  :  1  <  i  <  n}  be  a  sequence  of  independent,  non-identical, 
lattice  random  variables.  Let 


5n  = 

^=1 


and  suppose  that 


E  [WP]  =  0,  E  {WP) 


=  Si 


E 


n\3 


iwp) 


=  /i?. 


Define 


=  Ie  [52] 

n 

fin  =  -E  [53] 


i=l 


Finally,  define  the  characteristic  functions  pn{t)  =  E  [e*^'^"]  and  = 

E  . 

The  theorem  requires  that  the  sequence  of  distributions  satisfy  the  fol¬ 
lowing  conditions: 


Condition  3.1.  1.  limcr^  =  a,  where  0  <  <7  <  oo. 


2.  \\m.fin  =  fi,  where  G  M. 

3.  There  is  0  <  C  <  oo  such  that  for  any  n  and  all  1  <  i  <  n,  sf  <  C 
and  I /if  I  <  C. 

4-  There  is  0  <  C  <  oo  such  that  for  any  n  and  all  1  <  i  <  n,  K{Wp)^  < 
C  (without  loss  we  assume  C  is  the  same  constant  as  in  part  3). 


5. 


For  any  (5  >  0,  there  exists  0  <  6  <  1  so  that  for  all  t  G  [— vr,  —(5]  U  [(5,  vr] 
and  all  n, 

\(l>n{t)\  <  6". 
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Remark  3.1.  Parts  1  to  4  of  the  condition  are  mild  and  easy  to  check. 
However  part  5  can  be  nontrivial.  It  is  easy  to  check  when  cfn  is  of  the  form 
[0(t)]”,  corresponding  to  identically  distributed  random  variables.  However, 
for  non-identical  random  variables  it  need  not  hold.  We  thus  give  a  simple 
sufficient  condition  for  part  5  which  is  adequate  for  the  present  problem. 

Condition  3.2.  For  any  5  >  0  there  exist  constants  c  G  (0,1),  C  G  (0,1), 
and  for  each  n  G  N  there  exists  a  subset  A„  of  {1,2, ...  ,n}  such  that  |A„|  > 
fn  and 


|<('j*(i)|  <  c,for  all  j  G  A„  and  any  t  G  [— vr,  —(5]  U  [5,  vr]. 


It  is  straightforward  to  show  that  Condition  3.2  implies  part  5  of  Con¬ 
dition  3.1. 

The  following  theorem  gives  a  refined  expansion  of  the  central  limit  theo¬ 
rem  for  non-identical  independent  lattice  random  variables.  In  [9,  Page  531, 
Chapter  XVI] ,  approximation  theorems  are  proved  for  i.i.d  random  variables 
(lattice  and  non-lattice),  and  an  outline  is  given  for  the  non-identical,  non¬ 
lattice  case.  However,  even  in  the  non-identical  lattice  case  the  results  in  [9] 
take  us  most  of  the  way,  although  a  replacement  for  the  argument  leading 
to  the  estimate  (4.13)  in  [9,  Page  541]  is  needed,  see  our  condition  below. 
The  proof  of  the  following  theorem  can  be  found  in  [20] . 

Theorem  3.3.  Suppose  Wp,i  =  l,...,n  are  independent  lattice  random 
variables,  with  lattice  spacing  h,  first  three  moments  (0,  sf,  pdf)  and  bounded 
fourth  moments  satisfying  Condition  3.1.  Let  Hn{x)  be  the  distribution  func¬ 
tion  of 


where  naf  =  (x)  be  the  distribution  function  which  is 

obtained  by  interpolating  Hn{x)  linearly  through  the  midpoints  of  the  lattice. 
Let  J\[{x)  be  the  cumulative  distribution  function  of  the  standard  V(0, 1) 
distribution  and  91  be  the  corresponding  density.  Then 


uniformly  in  x. 


Observe  that  by  construction  Hn{x)  =  Hn{x)  at  midpoints  of  their 
lattice,  which  has  spacing  h/  {an\^). 
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3.2  A  Limit  Theorem  for  the  Exponents 

Recall  the  definitions  of  p  and  pn  in  Sections  2.1  and  2.2,  where  p  is  the 
unique  positive  root  to  (2.3)  and  is  characterized  by  (2.10).  Our  first 
goal  is  to  show  that  lini„_>oo  n{pn  —  p)  exists  and  identify  the  limit.  Recall 
from  the  discussion  below  (2.3)  that  p^  <  1.  It  is  also  worth  recalling  that 
the  assumption  that  0  >  ^  is  harmless,  in  that  9  =  ^  and  9  <  ^  correspond 
to  situations  that  are  easy  to  analyze  and  vacuous,  respectively.  See  Remark 
2.1. 


Lemma  3.4.  Assume  ^  9  >  ^,  so  that  pn  is  well  defined  by  (2.10)  and 

p  is  well  defined  in  (2.3).  Then 


K 

Pn-  P= - Vo 

n 


where 

K  =  i _ _ . 

2  (1  -  pO  log(l  -  pO  +  pi 

Proof.  Define  f{p,x)  =  and  recall  that  g"  =  {j  —  l)/n  for  j  = 

1, . .  .,ni.  From  the  definition  of  p  and  using  the  fact  that  ni  is  an  inte¬ 
ger,  it  follows  that 


rr 


1 


-dx 


'0 

E 

i=l 
ni 


-  px 


f{p,x)dx 

ni 

I 

j  =  l  ■^‘>3 


1  ^  ^  /*9  _j_l 

- f{p, Qj)  +  Y I 

”  J=l  3=1 


By  elementary  arguments  using  Taylor’s  Theorem  the  last  expression  is 


Mn{i,p)ni 
6n^  ’ 


where  Mn{i,  p)  accounts  for  the  remainder  terms  and  M  =  sup„  Mn{i,  p)  < 
oo.  Since  (2.10)  can  be  expressed  as 


9  = 


1 

n 


ni 


i=i 


14 


we  obtain 


'-EifiP.  9?)  -  /{/>».  9?))  +  E  •  9")  +  O 


=  0. 


(3.1) 


We  pause  to  estimate  the  difference  pn  —  p-  Recall  that  <  1.  Fix  p  >  p 
so  that  <  1,  and  denote 

I{a)  =  /  f{a,x)dx. 

Jo 

Then  I{p)  =  6.  Let  the  solution  for  I  (a)  =  9  he  denoted  pg,  and  observe 
that  I  {a)  is  monotone  increasing  and  continuous  in  a.  Thus  for  sufficiently 
small  e  >  0,  pg+^  <  p.  Let 


Sn{a)  = 

i=i 

Then  Sn{pn)  =  9.  Also  observe  that  Sn  is  monotone  increasing  and  con¬ 
tinuous  in  a  as  well.  Since  /(ce,  x)  is  monotone  increasing  in  x,  for  any 
p  <  a  <  p 

Sn{a)  <  I{a)  <  Sn{a) +  -  ^  ,  (3.2) 

n  1  — 

where  the  first  inequality  uses  that  Sn{a)  is  a  lower  Riemann  sum,  and  the 
second  uses  that  Sn{a)  +  ~  n  upper  Riemann  sum.  Since  (3.2) 

implies  S'n(p)  <  I {p)  =  9,  pn  >  p-  Inserting  a  =  pe+e  into  (3.2)  gives 

'S’n(p0+e)  <  0  +  e  <  Sni^pgj^f)  H - - - - - . 

n  1  -  ?P0+€ 

For  e  >  0  small  enough  that  p^+e  <  P, 

|0  +  e-54pe+,)|  < 

n 

where  Ai  =  It  follows  then  limsupp„  <  pe+e,  and  since  e  >  0  can  be 

arbitrary  small  fim  sup  p„  <  p.  We  have  already  shown  p„  >  p,  and  thus 


lim  Pn  =  p. 
n—^oo 


We  have  shown  that  for  all  sufficiently  large  n  p  <  pn  <  p-  Inserting 
a  =  Pn  into  (3.2)  and  using  that  Sn{pn)  =  &  =  I{p)^  one  obtains 

—  >  |l'(Pn)  -  Iip)\  >  -43  \pn  -  p\  . 
n 
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Here  As  >  0  is  a  lower  bound  on  the  derivative  of  I  with  respect  to  p  G  [p,  p] . 
It  follows  that  lim  sup„_,j,Q  n\p  —  pn\  <  oo.  Returning  to  equation  (3.1)  and 
applying  a  Taylor  series  expansion  with  respect  to  p  to  the  first  term, 

■  iz  (^) = 0- 

j=i  j=i  \ '' 

Since  limsup^^g^^  n\p  —  pn\  <  oo,  we  may  consider  a  subsequence  such  that 
lim^rifc  (p  —  pru,)  =  K.  Dropping  the  k  subscript  to  simplify  the  notation, 
the  last  display  becomes 


ni 


d 


K 


1  ^  a 


E  °  (s? j  j  +  2^;  E  .?)+o  ( - )  =  0,  (3.3) 

.7  =  1  \ 


We  conclude  that 


ni 


ni 


„S"  E  9?)  +  i  E  "")  =  »■ 

i=i  ^  i=i 


Again  using  that  <  1, 


and 


sup  sup 
0<i<< 


sup  sup 
0<i<< 


Afip.gP-Afip.x) 


^  iff)- -^fip.x) 


dx 


0. 


Hence  by  the  Lebesgue  Dominated  Convergence  Theorem,  each  of  the  Rie- 
mann  sums  in  (3.3)  converges  to  the  corresponding  integral,  therefore 

d  _  1  /■«  9 


-K 


r  d  I  n  d 

i  ^/{p,x)dx=0. 


This  is  just 


-K 


f 

Jo 


X 


-,dx  + 


1  /  1 


/o  (l-px)2  2Vl-pe 

and  computing  the  integral  gives 

1  p3^ 


-1=0, 


K  = 


2(1-p0  log(l-pO  +  Pr 

Since  this  limit  is  independent  of  the  subsequence  the  convergence  is  proved. 

□ 
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Next  we  state  a  result  on  the  on  the  asymptotics  of  Jn{0)  — 

Theorem  3.5.  Assume  ^  <  1  and  9  >  ^,  define  p  to  he  the  unique  positive 
root  of  (2.3),  and  define  Jn  and  J  by  (2.8)  and  (2.2),  respeetively.  Then 

lim  n(J(0)  -  Jn{e))  =  ^log  . 

n— >oo  2  \  I  —  ^  J 

Proof.  For  a  >  0  let  F{a)  =  F(loga)  and  Fn{a)  =  Fn(loga),  where  F  and 
Fn  are  dehned  in  (2.6)  and  (2.7),  respectively.  From  the  dehnitions  of  J  and 

Jni 

J{9)  =  {\ogp)e-F{p) 

Jn{0)  =  {log  Pn)0  -  Fn{pn)- 

It  follows  that 


n{J{e)  -  .Jn{9))  =  n{\og  p- log  pn)9  -  n{F{p)  -  Fn{pn))- 
By  Lemma  3.4 

log  p  -  log  Pn  =  -{pn-  P)  ■-  +  0{pn  “  p) 

P 


K1 
n  p 


+  o 


Also 


(3.4) 


(3.5) 


F{pn)-F{p)  =  {pn- p)F'{p)  +  o{pn- p) 

T' 


-F'{p)  +  o 
n 


n 


We  continue  to  estimate  the  difference: 


(3.6) 


F{pn)-Fn{pn)  =  -(l-01og(l-a  +  ^— ^log(l-M) 

Pn 

i=i  i=i 


/■?  r? 

/  log(l  —  x)dx  —  /  log(l  —  PnX)dx 

Jo  Jo 


n 


log(l  -  q^)  +  -Y  ^°S(1  -  PnQj)- 


i=i 


i=i 
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With  f{x)  =  log(l  —  x)  and  gn{x)  =  log(l  —  pnx)  the  last  equation  becomes 


F{pn)  -  Fn{pn) 
ni 


'  U{x)  -  f{qj))dx  -Y  I  ^3n{x)  -  gn{qj))d: 
— 1  ■^<7.  1 


/  /T'*' 

i=i  ^7  -1 

j=l  J=1 


2n2 


Therefore 

»{i^{P«)  -  Fn(Pn))  =  ^  E  /'(«")  -  ^  E  a'M)  +  »(!)■  (3.7) 

i=i  j=i 

Since  ^  <  1,  by  the  Lebesgue  Dominated  Convergence  Theorem, 

lim  n{F{pn)  -  Fn{pn))  =  ^  /  f'{x)dx  -  ^  /  gXx)dx 
n^oo  2  Jq  2  Jq 

i.e., 

lim  n{F{pn)  -  Fn{pn))  =  ^log(l  -€)-  ^log(l  -  pi).  (3.8) 

n^oo  Z  Z 

Now  insert  (3.8),  (3.6)  and  (3.5)  into  (3.4)  to  obtain 

lim  n(J(0)  -  ue))  =  --e  -  Jlog(l  -  e)  +  Jlog(l  -  pO  +  KF\p). 
n-^oo  p  Z  Z 

Since  9  =  F'{logp)  and  F'{logp)  =  F'{p)p,  therefore  F'{p)  =  Ojp.  Thus 
lim  n{J{9)  -  Jn{9))  =  ^log  . 

n^oo  2  \  V  —  J 

□ 


Note  that  the  value  of  K  is  not  used  in  the  proof  at  all,  only  the  existence 
of  the  limit.  Also  notice  that  because  of  the  singular  behavior  of  log(l  —  ^) 
at  ^  =  1,  we  will  have  to  separate  the  case  when  ^  =  1.  In  fact  when  1^  =  1 
we  have  the  following  modified  version  of  Theorem  3.5. 

Theorem  3.6.  Assume  ^  =  1  and  6  >  ^,  define  p  to  be  the  unique  positive 
root  of  (2.3),  and  define  Jn  and  J  by  (2.8)  and  (2.2),  respectively.  Then 

lim  ( n{J {9)  -  Jn{9))  -  log n)  =  ^  log(27r(l  -  p)). 
n— ^■oo  y  Z  J  Z 
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Proof.  First  observe  that  (3.4),  (3.5),  (3.6)  and  F'{p)  =  6 jp  still  hold. 
Thus 

n{J{0)  -  Jn{0))  -  ^logn  =  n{log  p  -  log  pn)0  -  n  (^F{p)  -  Fn(Pn))  -  ^logn 

(3.9) 

and 


log  p  -  log  pn 

F{pn)  -  F{p) 


K1 

- h  o 

n  p 


K6 

- h  O 

n  p 


The  last  two  displays  imply 

i™)  +  n  (^(Pn)  -  ^(p)))  =  0. 


(3.10) 


The  only  discrepancy  occurs  when  we  compute  Fn{pn)  —  F{pn)-  Letting 
gn{x)  =  log(l  -  pnx), 

Fn{Pn)  -  F{pn) 

n  n  _ 

= - ^°S(1  -  QjPn)  +  -Y1  ) - —  “  P^) 

=  E<W)i  +  (i  +  ^E‘»i5(i>”))+'’(^)'  p-ii) 

j=i  y  j=i  j  \  / 


The  last  equality  is  because  we  interpret  log(l  —  Pn)  as  gn(x)dx  +  l 

and  then  use  a  Taylor  expansion  on  gn{x)  as  was  done  to  obtain  (3.7).  Since 
P]  =  {n-j  +  l)/n,j  =  1, .  .  .,n. 


1 

n 


i=i 


By  Stirling’s  formula,  [8,  Page  54,  (9.15)] 


n  \ 


{ log\/2 


7rn  + 


1 


(12n  +  1)  y  n 


<  -^iog(p")  +  i  <  -  hogV^+  —  ] , 

j=i  \  / 
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and  thus 


1  ^  1 

- -logn  1  = -log(27r).  (3.12) 

By  the  Dominated  Convergence  Theorem 

"'ll  1 

lim  -  9{^))  =  -  p).  (3.13) 

i=i 

Now  combining  (3.12)  and  (3.13)  with  (3.11)  gives 

i™,  (^n(/On)  -  F{Pn)'j  -  ^  log =  ^log(27r(l  -  p)). 

The  argument  is  completed  by  combining  the  last  display,  (3.10),  and  (3.9). 

□ 


lim 

n— >cxD 


n 


n 


n 

E 

i=i 


logp"  +  1 


3.3  Proofs  of  the  Main  Theorems 


In  this  subsection  we  give  the  proofs  of  Theorems  2.1  and  2.2.  For  reasons 
outlined  in  the  introduction,  we  start  with  Theorem  2.1.  Following  [3],  we 
first  represent  the  probabilities  using  a  change  of  measure  suggested  by  the 
large  deviations  analysis.  This  will  exhibit  each  probability  as  the  product 
of  an  exponential  and  an  integral.  The  exponential  represents  the  difference 
between  the  large  deviation  approximation  and  an  exponential  term  com¬ 
ing  from  our  particular  change  of  measure,  and  can  be  approximated  using 
Theorem  3.5.  To  approximate  the  integral  we  use  the  refined  CLT  stated  in 
Theorem  3.3,  and  the  final  result  just  combines  these  two  approximations. 


Proof  of  Theorem  2.1,  (2.13),  (2.14).  Recall  that  <  1  and  p„  — >  p. 
Thus  for  sufficiently  large  n  the  independent  random  variables 


\rn 

y\  ■ 

3 


GiPnq'i 


are  well  defined.  Let 


=  IE 


1  -  PnQj 


and  let  —  0”  be  the  corresponding  random  variables  centered  at  0. 

Recalling  (2.10)  and  (2.11),  we  find  that 


E -ii  -  ne 

vWf 


Un  = 


(3.14) 
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has  0  mean  and  unit  variance.  Let  Hn{u)  =  lP{C/n  <  "«}•  C/n  is  a  lattice 
random  variable  with  lattice  points 


—  nO  —  nO  -\-l  —  nO  +  2 

Let 

dn  =  -iL= 

denote  the  lattice  step  size,  and  observe  that  0  is  a  lattice  point  because 
nO,  are  integers  and  9  >  ^. 

By  expressing  <  0}  and  >  9}  in  terms  of  Un  via  the 

change  of  measure  that  relates  the  distribution  of  X”  to  that  of  X",  we 
obtain 

P  <  0}  =  f  exp{—anU\/7ian)dHn{u)  (3.15) 

J{u<0} 

P  >  0}  =  /  exp{—anU\/7ian)dHn{u)  (3.16) 

J{u>0} 

(see  the  Appendix  for  the  details  of  this  calculation).  Therefore 


v/^  P  {T”(e)  <  9}  f  ^  exp(-a>\/^u„)dF„(u) 

J{u<0} 


(3.17) 


n  P{F”(e)  >  9}  f  ^  exp(-a>\/^u„)dF„(u). 

J{u>Q} 

(3.18) 


From  now  on,  let  us  focus  on  the  proof  of  (2.13).  Denote 

An  =  n{J{9)  —  Jn{9))  and  Bn  =  y/n  exp{—anU\/nan)dHn{u). 

J{u<0} 

(3.19) 

First  notice  that  in  (2.13)  we  are  in  the  case  ^  <  0,^  >  1  —  e“®.  As  was 
discussed  below  (2.8),  this  implies  a*  <  0.  Since  Hn{u)  is  the  cumulative 
distribution  function  of  the  lattice  variable  Un,  the  integral  in  Bn  can  be 
written  as 

0 

Bn  =  Vn  exp{-ank)  [Hn{kdn)  -  Hn{{k  -  l)dn))] 

k=—oo 

=  \/re  I  ^  Hn{kdn)  [exp(-Q;*  A:)  -  exp(-Q;*  (A:  +  1))]  +  Hn{0) 

\k=—oo 


21 


Since  a*  <  0, 


-1 

Bn  =  Vn  ^  {Hn{kdn)  -  Hn{0))  [exp(-a*/c)  -  exp(-a*  (A:  +  1))]  . 
k=—oo 


By  definition  (2.9)  pn  =  exp(a*),  and  therefore 

-1 

Bn  =  V^  Y.  i^nikdn)  -  F„(0)) 

k=—oo 

oo 

=  V^Y  -  Hn{-kdn))  (p^'  -  p^r)  ■ 

k=l 

Let  A„  =  l/{2-^na‘^)^  and  Hn{u)  be  the  distribution  function  obtained 
from  Hn{u)  by  linear  interpolation  through  the  midpoints  as  stated  in  The¬ 
orem  3.3.  Therefore  if  u  is  one  of  the  midpoints  of  the  lattice  then  iLjf  [u)  = 
Hn(u)  by  construction.  Also,  since  Hn  is  piecewise  constant  and  jumps  just 
at  the  lattice  points,  Hn{kdn)  =  Hn{kdn  +  A„)  for  any  k  ^  "L.  By  setting 
=  kdn  +  An  and  inserting  this  into  the  formula  for  Bn,  one  obtains 

OO 

Bn  =  V^Y  W)  -  H*{ulk))  -  Pn)  ■  (3.20) 

k=l 


We  now  apply  the  normal  approximation  to  Hn{u)  as  given  in  Theorem 
3.3.  In  the  notation  of  that  theorem  WJ^  =  Z^,  k  =  1,  •  •  •  ,  and  WJ^  =  0 
otherwise.  Parts  3  and  4  of  Condition  3.1  are  satisfied  as  are  lattice 
random  variables  with  uniformly  bounded  fourth  moments: 


E 


<  C  <  oo. 


This  uniform  bound  follows  from  the  fact  that  for  any  e  £  (0,1  —  p^)  there 
is  a  uniform  bound  on  the  failure  probabilities  PnQk  <  P?  +  e  <  1  for  all 
sufficiently  large  n.  Since  the  have  0  mean, 

1  <  .  . 

k=l 

It  is  readily  verified  that 


ex;(%) 


2 


=  na 


2 

ni 
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where  cr^  is  obtained  through  (2.11).  Moreover  cr^  — >  =  F" [a*)  [see 

Section  2.2].  It  is  also  readily  verified  that  and  in  fact 

— >  F'"{a*).  Hence  parts  1  and  2  of  Condition  3.1  are  also  satisfied. 
Verification  of  part  5  follows  from  Lemma  A.l,  which  is  stated  and  proved 
in  the  appendix. 

Applying  Theorem  3.3  and  gives 


+  [(1  -  -  (1  -  +  o 


where  o(l/y^)  is  uniform  in  A;  G  Z. 
Observe  that  for  fixed  A:  G  N, 


lim 

n— ^■oo 


1  k 


(3.21) 


and  that  the  left  hand  side  is  dominated  by  A  A:  for  some  K  <  oo.  In 
addition, 


T  1(1  _  {„5)2)9i«)  -  (1  - 

\/n 


< 


Kk^ 

y/n 


for  some  K  <  oo  and  without  loss  we  assume  it  is  the  same  K  used  to  bound 
the  normal  distribution.  Also,  for  each  fixed  A:  G  Z, 


as  n  ^  oo.  Finally,  observe  that  Pn  ^  P  implies  for  any  A;  G  N, 


^k-l 


Pn 


P 


k-1 


P 


and  Pn  <  1  for  sufficient  large  n.  By  the  Dominated  Convergence  Theorem 


lim  Br, 

n-^oo 


OO 


E 


^  -.k- 


\/^  cr 


1 

\/^it(1  —  p) 


(3.22) 


Applying  Theorem  3.5  to  in  (3.19)  and  combining  the  estimate  (3.22), 
we  finally  have 


lim  VnlP{V"(e)  < 

n— ^■oo 


W  1  A  /iZZ 
V2t^  ~  pJ  \j 
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We  next  prove  Theorem  2.1,  (2.14).  The  idea  and  technique  of  the  proof 
is  largely  the  same  as  the  proof  for  (2.13).  In  this  case  ^  <  1  —  e“®,  and  as 
remarked  below  (2.9),  /?„  >  1  for  all  sufficiently  large  n.  In  formula  (3.18), 
we  let 


Omitting  a  few  details,  this  can  be  rewritten  as 


Bn  =  Yl  V^{Hn{kdn)  -  Fn(-dn))  {Pn’'  " 

k=0 

oo 

=  -  H*{ul,))  {p->^  - 

k=0 

Applying  Theorem  3.3  in  a  similar  manner  as  in  the  last  case,  the  Dominated 
Convergence  Theorem  gives 


lim 

n^oo 


OO 


E 


k  +  1 


P 

'J^a{p  —  1) 


Applying  Theorem  3.5  to  the  exponential  part  of  (3.18)  finishes  the  proof  of 
Theorem  2.1,  (2.14).  □ 

Before  analyzing  the  final  case  ^  =  1  for  the  filling  process,  we  prove  the 
refined  asymptotics  for  the  occupancy  process  when  ^  <  1. 

Proof  of  Theorem  2.2,  (2.16),  (2.17).  Since  (2.4)  holds,  (2.13)  implies 
(2.16).  To  analyze  (2.17)  we  use  (2.5),  which  states  that  V2  is  the  union  of 
W2  and 

cn  =  ^  0  ^  , 

From  (2.14)  we  already  know  that  P{iy2}  has  yprefactor 


1  ^  p  A  liZR, 

\/2'Ka'^  \P  ~  ^  J  \j 

Thus  to  show  (2.17)  we  must  prove  that  has  |-prefactor 

1  ^  /"O  /i-pg 
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with  the  same  exponent  J{0).  From  the  definition  of  it  is  easy  to 

verify 

F{x:i^^,>k}  =  e- 

With  G{x)  =  <  x},  we  can  represent  PjC”}  as 

P{C'"}=  [  C^-^'^dGix). 

J  {x<9} 

By  the  same  change  of  measure  argument  as  used  in  (3.17),  and  with 
the  cumulative  distribution  function  of  Un  defined  in  (3.14) 


J  {ii<0} 

=  e-nXi9)  f  dHn{u).  (3.23) 

J  {w<0} 

Replacing  exp  a*  by  pn,  in  (3.22)  we  showed  that 

lim  ^/^  f  (pn)““^'""  dHniu)  =  ^ - -. 

J{u<o}  V27r(T(l  -  p) 

This  limit  continues  to  hold  if  pn  and  p  are  replaced  by  pn^  and  p^,  so  long 
as  <  1.  Therefore 


lim  y/n 

n— >oo 


{tt<0} 

1 


dHniu) 


-  lim  y/n  {Hn{0)  -  Hniuh)), 


y/^ail  -  pi) 

where  is  defined  in  (3.20).  Using  (3.21)  and  the  refined  normal  approx¬ 
imation,  Theorem  3.3,  we  have 


lim  y/niHn{H)  -  Hn{u^i))  = 


'TTa 


This  shows  lim^^oo  Vn {pnO  dHniu)  =  Applying 

Theorem  3.5  to  the  exponential  part  of  (3.23)  shows  that  P{C”}  has  |- 
prefactor  ^  with  the  same  exponent  J(0),  and  hence  fin¬ 
ishes  the  proof  of  Theorem  2.2,  (2.17).  □ 


Finally,  we  will  prove  the  ^  =  1  case  in  Theorem  2.1  and  Theorem  2.2, 
i.e.,  (2.15),  (2.18).  Because  of  (2.4)  we  need  only  prove  (2.15)  and  (2.18) 
will  follow. 
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Proof  of  Theorem  2.1,  (2.15).  The  only  difference  between  this  case  and 
the  proof  of  (2.13)  is  that  we  can  no  longer  use  Theorem  3.5  to  calculate  the 
limit  of  An-  We  rewrite  the  representation  for  P{y”(l)  <  6}  slightly  as 

P{y"(l)  <  6}  [  exp{-a*^u./^an)dHn{u), 

J{u<0} 

and  observe  that 


is  still  valid.  By  Theorem  3.6 

^n{j(e)-Me))  _ 

lim  - — - =  Y^27r(l  -  p). 

n— >oo  yjn 

Putting  these  together  gives 


lim  P{y"(l)  <  6]  =  — ; - 

n->oo  (JyJX  —  p 

which  completes  the  proof. 

□ 


4  Numerical  Results 

The  following  tables  and  figures  compare  the  refined  asymptotic  results  of 
Theorems  2.1  and  2.2  with  exact  results  using  the  inclusion-exclusion  prin¬ 
ciple  and  numerical  estimates  obtained  by  importance  sampling.  In  all  cases 
importance  sampling  was  conducted  for  10^  trials.  In  the  graphs  approxi¬ 
mations  based  on  unrefined  large  deviation  asymptotics  are  also  presented. 

Table  1  presents  results  for  the  filling  process,  case  (2.14),  for  =  0.5,  9  = 
1.2,  and  a  range  of  values  for  the  scale  parameter. 

As  the  table  shows  the  refined  asymptotic  tends  to  overestimate  the 
probability  but  with  a  percentage  error  that  decreases  as  n  increases.  At 
n  =  10  this  is  well  over  50%,  however  by  n  =  20  it  has  fallen  to  under  30% 
and  by  n  =  100  it  is  only  6%. 

In  Figure  1  we  depict  the  combinatorial  probability  again  with  n  =  100, 
0  =  1.2  as  above,  and  with  ^  varying.  The  probability  estimated  from  the 
large  deviations  exponent  alone  is  also  shown.  This  overestimates  the  true 
value  by  roughly  a  factor  of  10. 
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n 

Combinatorial 

Rehned  Approx. 

Imp.  Sample 

10 

7.55  X  10"^ 

1.22  X  10"^ 

7.43  X  10"^ 

20 

9.56  X  10-^ 

1.24  X  10"^ 

9.56  X  lO-"^ 

50 

2.10  X  10"“ 

2.35  X  10"“ 

2.08  X  10"“ 

100 

9.73  X  10-^^ 

1.03  X  10"^^ 

9.71  X  10"^^ 

200 

2.74  X  10-iy 

2.82  X  10-iy 

2.68  X  10-iy 

Table  1:  Estimates  for  P  {T”(^)  >  0}  ,  ^  =  0.5,  9  =  1.2. 


The  combinatorial  probabilities  were  obtained  as  follows.  Recall  that 
T”(C)  >  ^  is  the  event  that  at  least  n6  balls  are  required  to  hll  urns.  We 
decompose  this  according  to  whether  or  not  exactly  nO  balls  are  required. 
Thus  we  can  write 


{Y-{O>0}  =  {Y-{O>O}uAli 

where  is  the  event  that  the  n^th  urn  to  be  hlled  is  hlled  with  the  ndth 
ball  that  is  thrown.  Recall  the  notation  that  we  used  in  Section  2.1.  Tq  (t) 
is  the  fraction  of  empty  urns  after  [tn\  balls  are  thrown.  Setting  r  =  n6, 
m  =  n(l  —  the  probability  of  A^^  is 

p/rg  ri±i\ . 

I  \  n  J  n  j  n 

Meanwhile  the  event  {T"(C)  >  9}  can  be  written  as  Tq  (^)  >  The 
“exact”  calculation  of  the  quantities  P  {Tq  P  {Tq 

can  then  be  obtained  using  the  well  known  method  of  inclusion  and  exclusion 
as  described  in  [8,  Chapter  II. 11]  and  as  denoted  there  [8,  Chapter  II,  (11.9)] 

a:^(r,n)  =  p|r((  .  (4.1) 

Table  2  is  for  the  case  when  exceptionally  few  balls  are  needed  to  hll 
a  given  fraction  of  urns,  which  corresponds  to  (2.13)  in  Theorem  2.1.  The 
blanks  indicate  cases  when  the  combinatorial  expression,  as  computed,  gave 
obviously  incorrect  values  which  resulted  from  rounding  errors.  The  pat¬ 
tern  of  results  is  similar  to  that  of  Table  1  with  the  rehned  approximation 
overestimating  the  underlying  probability  but  with  a  decreasing  percentage 
error  as  n  increases. 

We  now  turn  to  the  case  of  hlling  all  the  urns.  Theorem  2.1,  (2.15). 
Figure  2  shows  results  for  the  probability  of  hlling  n  =  50  urns  with  9  G 
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^  (Fraction  urns  to  be  filled) 

Figure  1:  Estimates  for  P{y"'(^)  >  6}  and  the  “exact”  result. 


n 

Combinatorial 

Refined  Approx. 

Imp.  Sample 

10 

1.53  X  10-1 

1.73  X  10-1 

1.52  X  10-1 

20 

2.47  X  10"^ 

2.64  X  10-^ 

2.46  X  10-^ 

50 

1.61  X  10-4 

1.68  X  10-4 

1.63  X  10-4 

100 

- 

5.58  X  10-« 

5.44  X  10-« 

200 

- 

8.72  X  10-1^ 

8.65  X  10-1^ 

Table  2:  Estimates  for  P  =  0.8,  6  =  1. 


[1.4,  2.0].  Sample  results  are  given  in  Table  3  for  the  probability  of  filling 
urns  with  only  half  as  many  additional  balls.  Blanks  indicate  rounding  errors 
in  the  combinatorial  calculation  as  before. 

For  Theorem  2.2  we  take  only  the  case  of  (2.17)  as  the  other  two  results 
are  identical  with  the  corresponding  ones  for  Theorem  2.1.  The  refined  ap¬ 
proximation  is  for  the  probability  that  there  are  at  least  (1  —  ^)n  urns  empty 
after  9  balls  per  urn  have  been  thrown.  By  definition  this  is  determined  as 
Xm{T,n)  as  we  discussed  before  in  (4.1).  Furthermore  this  probability  is 
greater  than  the  probability  that  it  takes  nO  balls  to  fill  urns  as  shown 
in  (2.5). 

Our  results  are  depicted  in  Figure  3,  again  for  n  =  100,  9  =  1.2,  as  in  the 
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e 

Figure  2:  Results  for  P{y"'(^)  <  9}  the  refined  approximation  and  “exact” 
results,  ^  =  1. 


case  of  Figure  1.  These  results  are  given  in  Table  4.  Comparison  of  Tables  1 
and  4  shows  that  the  probabilities  differ  by  a  significant  factor,  e.g.,  roughly 
a  factor  of  4  in  the  case  n  =  50. 
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n 

Combinatorial 

Refined  Approx. 

Imp.  Sample 

10 

4.60  X  10"^ 

4.66  X  10"^ 

4.60  X  10"^ 

20 

1.32  X  10"^ 

1.33  X  10"^ 

1.32  X  10"^ 

50 

3.06  X  10-« 

3.07  X  10-« 

3.04  X  10-« 

100 

- 

5.75  X  10"^*^ 

5.76  X  10-^“ 

200 

- 

2.02  X  10-^1 

2.02  X  10-^1 

Table  3:  Estimates  for  P  {T"(^)  <  0}  ,  ^  =  1.0,  0  =  1.5. 


Figure  3:  Estimates  for  P{rQ(0)  >  1  —  ^}  and  the  “exact”  result 


A  Appendix 

We  begin  by  showing  that  Condition  3.2  holds. 

Lemma  A.l.  The  sequences  of  random  variables  Zlj  meet  Condition  3.2. 
Proof.  In  our  model 

j=l 

where  Z”  =  —  0”  and 

a;  ~  G{pnq^) 
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n 

Combinatorial 

Refined  Approx. 

Imp.  Sample 

10 

4.49  X  10"^ 

5.01  X  10"^ 

4.49  X  10"^ 

20 

4.79  X  10"^ 

5.11  X  10"^ 

4.77  X  10"^ 

50 

9.39  X  lO-*^ 

9.67  X  lO-*^ 

9.41  X  lO"*^ 

100 

4.19  X  10"^^ 

4.25  X  10"^^ 

4.18  X  10-^*^ 

200 

1.15  X  10-i« 

1.16  X  10-i« 

1.15  X  10-i« 

Table  4:  Estimates  for  P  {Tq  (0)  >  1  —  ^  =  0.5,  6  =  1.2. 


and 


9] 


1 


1  -  PnQi 


So  we  can  extend  the  random  variables  to  all  1  <  j  <  n  by  setting  =  ZJ* 


when  j  and  =  0  when  <  j  <  n.  We  determine  (t)  =  E 


for  each  Z”  as 


^lit)  = 


e  ^  ^  ^  PnQj 

1  -  fl  - 


for  1  <  j  <  ^n. 


Thus 


\4>m  = 


PnQi 


1  -  e*‘(l  -  pnq"-) 

Therefore  it  suffices  to  show  that  for  all  1  <  j  <  and  t  G  [— vr,  —(5]  U  [(5,  vr] 
there  is  c  >  1  so  that 

1  -  e*‘(l  - 


PriQi 


>  c. 


(A.l) 


Let  y”  =  then  y”  >  Since  Pn  ^  P  and  <  1  we  can  assume 

y^  >  y  >  1,  where  y  is  some  constant.  Now  we  have 


1  -  e*‘(l  -  pnqD 


Pnq- 


Define  /(t,  x)  =  \x  —  e**(x  —  1)|^,  so  that 

/(t,  x)  =  —  2  costx{x  —  1)  +  (x  —  1)^. 
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Since  t  G  [— vr,  —(5]  U  [5,  vr],  for  x  >  1  we  have 

f{t,  x)  >  2(1  —  cos  6)x‘^  —  2(1  —  cos  6)x  +  1  =  g{x). 

g{x)  is  monotone  increasing  on  [1/2,  oo),  and  since  y"  >  y  >  1  we  have 
y(y”)  >  y(y).  Therefore  when  t  G  [— vr,  —5]  U  [(5,  vr], 

2 

=  f{t,y^) 

>  g{yj) 

>  g{y)- 

Lastly  since  y  >  1  we  know  y(y)  >  y(l)  =  1.  Having  shown  (A.l),  it  follows 
that  our  model  satisfies  Condition  3.2.  □ 

We  next  show  that  the  change-of-measure  formulas  (3.15)  and  (3.16)  are 
true.  Since  they  are  similar,  the  proof  is  given  for  just  (3.15). 

Proof  of  (3.15).  We  recall  some  previously  used  and  also  introduce  some 
new  notation: 


1  -  e*‘(l  -  pnqf) 
PnQ] 


G„(x)  =p{y"(e)  <x} 

Hn{x)  =  ^{Un  <  u} 


Gnx)=r{xr<x} 

Hl^{x)=F[xr<x}. 


Recall  also  that  0/  =  IE  X^  and  that  ~  Denote  the  left  hand 

side  of  (3.15)  by  L  and  the  right  hand  side  by  R.  In  terms  of  this  notation 


R  = 


/ 


rii<<o 


exp  —ay 


/ 

\i=l 


HdHrK  +  e^).  (A.2) 


2=1 


Let  d6{-)  be  the  counting  measure  on  Z.  The  distribution  of  each  A”  has 
the  explicit  form 

dH^{z)={pnq?r-^{l-Pnq?)d6{z). 
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Thus 


i=l 


n  (1  -  Pnq:)d6{u^  +  e^) 


2  =  1 


ne-ni+  "ii  < 
—  Pn 


n (1  - Pnq?)]]  (llddK  +  0?) 


\i=l 


\i=l 


(A.3) 


where  the  last  equality  uses  0?  ~  Since  pn  =  e' 


exp  —Oj- 


E<))=P„- 


.,n 

i=l 


(A.4) 


By  definition  of  Jn{0)  in  (2.8) 

g-nJ„(0)  ^  ))] 


(ng  ng 

-n0a;  +  na;^  +  ^  log  (pf)  -  ^  log  (^1 

i=l  i=l 


q?e^^ 


Again  using  pn  =  e"",  this  expression  can  be  rewritten  as 

—1 
Pn 


,-nJn{e)  _ 


^-nO+n^  ( i-r<  „ 


nrii  (1  -  qiPn) 


(A.5) 


Inserting  (A.3),  (A.4),  and  (A.5)  into  (A. 2)  gives 


R  = 


:i<<0  i=i 


^{pnq^T^^'^-"d5{vd}  +  e^)) 


On  the  other  hand,  notice  that  by  definition 


L  = 


/ 


ni 

i=l 


x^<e 


ndGF(xF). 

2=1 
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Using  the  change  of  variables  uf  =  xf  —  Of , 


L  = 


/ 


Since  Gf  is  the  cumulative  distribution  function  of  X” 


L  = 


n  {pf{qfr^^^^-^d6{uf  +  Of))  =  R. 


2=1 


This  completes  the  proof  of  (3.15). 


□ 
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